Chapter 5: Sampling Distributions
Welcome to the online content for Chapter 5!
As always, I’ll assume that you’ve already read up to this chapter of the book and worked through the online content for the previous chapters. If not, please do that first.
As before, click the ‘Run Code’ buttons below to execute the R code. Remember to wait until they say ‘Run Code’ before you press them. And be careful to run these boxes in order if later boxes depend on you having done other things previously.
Standard errors
In the chapter, we wanted to test the null hypothesis that our sample of size 20 was a random sample from a normal distribution that had a mean of 170 cm and a standard deviation of 6 cm.
If we take samples of size 20 from this Normal distribution, their means, on average, will be 170 cm, but the standard deviation of those means (i.e. the standard error) will on average not be 6 cm but will be quite a bit smaller. In fact, if we take lots and lots of samples, the standard error will be\(\sqrt{20}\) times as small as the population standard deviation. The factor we divide by is the square root of the sample size. The bigger the sample size, the bigger its square root, and so the bigger the reduction in the standard deviation to get the standard error.
You could imagine that for a sample of size 1 the standard error would be \(\sqrt{1}\) times smaller, and since \(\sqrt{1}\) is equal to 1, this makes no difference.
So, our standard error is going to be the population standard deviation of 6 cm divided by \(\sqrt{20}\). If we want it to, R will work that out for us:
The sqrt function gives us the square root, and the / means ‘divided by’. We use brackets around the ‘20’ to say what we want square rooted. This allows us to distinguish, say, \(\sqrt{20+5}\), which is \(\sqrt{25}\) from \(\sqrt{20}+5\). These would have different values:
We always use brackets to specify the argument of the function - the value that we want the function to work on.
You can see from the previous box that the standard error of \(6/\sqrt{20}\) cm comes to about 1.34 cm, which is a lot smaller than 6 cm. For a sample bigger than 20 people, the reduction would be even more.
Hypothesis testing using a sample
We can use this value to test the null hypothesis that our sample of size 20 was a random sample from a normal distribution that had a mean of 170 cm and a standard deviation of 6 cm.
To see whether our sample mean height of 168 cm is small enough to reject this null hypothesis, we need to find out the percentage of this sampling distribution which is less than 168 cm. To do this, we need this code:
Other than adjusting the standard deviation to make it the standard error of the mean, nothing else changes from what we were doing in the previous chapter.
We could have put sd=1.341641, but letting R work it out from \(6/\sqrt{20}\) helps us to remember where that value came from, so is often preferable.
The output is the 6.8% value that I quoted in the chapter, which we doubled to obtain the \(p\) value of 13.6%. This was larger than 5%, so we didn’t reject the null hypothesis.
Type or paste into the code box below to try the same thing for a sample mean of 167 cm.
You’ll find that this sample mean is small enough to reject the null hypothesis, as we saw in the chapter.
Critical values
Critical values mark the boundaries between values for which we would reject the null hypothesis and values for which we wouldn’t. To find these critical values, we need to use the qnorm function, which give us the quantiles of Normal distributions.
In the online content for Chapter 3, we saw how to use pnorm to find the percentage of people shorter than a certain height. For example, this code told us that the percentage of people shorter than 168 cm was 36.9%:
The function qnorm answers the opposite question - what height are 36.9% of people shorter than?
The code to find this is:
And we get out of this the original height of 168 cm that we put into qnorm.
Mathematically, we could say that qnorm and pnorm are a pair of inverse functions, that do the opposite of each other.
For a 5% significance level, we want to know what height 2.5% of people are shorter than. It’s 2.5%, rather than 5%, because we’re doing a 2-tailed test, and we need to allow another 2.5% for the other (upper) tail.
Because we’re using the sampling distribution now, we need to make sure that our standard deviation is set to the standard error, which is \(6/\sqrt{20}\) cm. So, we do:
This gives us a critical value of 167.4 cm. Any sample mean height shorter than this critical value would lead to rejection of the null hypothesis.
To find the upper critical value, we need the lower.tail=FALSE option:
This gives us the upper critical value.
Confidence intervals
We use a similar process to find confidence intervals around our sample mean of 168 cm.
We want to create a confidence interval around 168 cm to show all of the possible values of the population height that we wouldn’t be able to reject in a hypothesis test, based on our observed sample mean of 168 cm. If we continue to use a 5% significance level, then we’ll be creating a 95% confidence interval.
We do exactly the same thing as we did for finding critical values, but with the sample mean of 168 cm, rather than the hypothesised population mean of 170 cm.
The lower limit of the confidence interval will be
and the upper limit of the confidence interval will be
Another way to find the upper limit, without having to type lower.tail=FALSE, is to change the quantile from 0.025 to 0.975, because 97.5% and 2.5% add up to 100%. This means that we’re asking for the height which 97.5% of means are shorter than, which is equivalent to asking for the height which 2.5% of means are taller than.
Either way, we obtain a 95% confidence interval around 168 cm that goes from 165.4 cm to 170.6 cm, and this matches what I stated in the chapter. Because 170 cm lies within this confidence interval, we don’t reject the null hypothesis.
Changing the confidence level
Let’s calculate a 99% confidence interval for the same data.
This time, we need to find quantiles for .995 and .005 (99.5% and 0.5%, which leave 0.5% at each end, making 1% in total). So, we need to code:
So, our 99% confidence interval around 168 cm goes from 164.5 cm to 171.5 cm. This is wider than the 95% confidence interval that we just worked out, because we’re more often unable to reject the null hypothesis when we have a lower (more demanding) significance level (1% now, rather than 5%).
Because 170 cm lies within this 99% confidence interval, we don’t reject the null hypothesis when using a significance level of 1%. (This necessarily happened - if we couldn’t reject the null hypothesis at the 5% significance level, then we certainly wouldn’t be able to at the more demanding 1% significance level.)
Changing the sample size
Finally, you’ll recall that towards the end of the chapter, I discussed what would happen if our mean of 168 cm had come from a sample of size 50, rather than of size 20. Being 2 cm below the hypothesised mean (170 cm) is much more startling if it’s from a larger sample than if it’s from a smaller sample. So, we expect that the 95% confidence interval around 168 cm would be much narrower for a larger sample. A larger sample allows us greater precision in estimating the population mean.
We can check this using the R code:
The population standard error has now changed from \(6/\sqrt{20}\) cm to \(6/\sqrt{50}\) cm. Dividing by a larger square root makes the standard error smaller.
So, a 95% confidence interval around 168 cm, if the data came from a sample of size 50, goes from 166.3 cm to 169.7 cm. This is much narrower than the previous 95% confidence interval that we obtained when the sample size was just 20. In particular, you’ll notice that 170 cm is no longer within the confidence interval, meaning that with a sample of size 50, and a significance level of 5%, we would now reject the null hypothesis.